50 research outputs found
Two derivations of Principal Component Analysis on datasets of distributions
In this brief note, we formulate Principal Component Analysis (PCA) over
datasets consisting not of points but of distributions, characterized by their
location and covariance. Just like the usual PCA on points can be equivalently
derived via a variance-maximization principle and via a minimization of
reconstruction error, we derive a closed-form solution for distributional PCA
from both of these perspectives.Comment: 4 pages, 1 figur
Argument Mining with Structured SVMs and RNNs
We propose a novel factor graph model for argument mining, designed for
settings in which the argumentative relations in a document do not necessarily
form a tree structure. (This is the case in over 20% of the web comments
dataset we release.) Our model jointly learns elementary unit type
classification and argumentative relation prediction. Moreover, our model
supports SVM and RNN parametrizations, can enforce structure constraints (e.g.,
transitivity), and can express dependencies between adjacent relations and
propositions. Our approaches outperform unstructured baselines in both web
comments and argumentative essay datasets.Comment: Accepted for publication at ACL 2017. 11 pages, 5 figures. Code at
https://github.com/vene/marseille and data at http://joonsuk.org
The Unreasonable Effectiveness of Random Target Embeddings for Continuous-Output Neural Machine Translation
Continuous-output neural machine translation (CoNMT) replaces the discrete
next-word prediction problem with an embedding prediction. The semantic
structure of the target embedding space (i.e., closeness of related words) is
intuitively believed to be crucial. We challenge this assumption and show that
completely random output embeddings can outperform laboriously pretrained ones,
especially on larger datasets. Further investigation shows this surprising
effect is strongest for rare words, due to the geometry of their embeddings. We
shed further light on this finding by designing a mixed strategy that combines
random and pre-trained embeddings for different tokens
Linguistic Harbingers of Betrayal: A Case Study on an Online Strategy Game
Interpersonal relations are fickle, with close friendships often dissolving
into enmity. In this work, we explore linguistic cues that presage such
transitions by studying dyadic interactions in an online strategy game where
players form alliances and break those alliances through betrayal. We
characterize friendships that are unlikely to last and examine temporal
patterns that foretell betrayal.
We reveal that subtle signs of imminent betrayal are encoded in the
conversational patterns of the dyad, even if the victim is not aware of the
relationship's fate. In particular, we find that lasting friendships exhibit a
form of balance that manifests itself through language. In contrast, sudden
changes in the balance of certain conversational attributes---such as positive
sentiment, politeness, or focus on future planning---signal impending betrayal.Comment: To appear at ACL 2015. 10pp, 4 fig. Data and other info available at
http://vene.ro/betrayal
Joint Dropout: Improving Generalizability in Low-Resource Neural Machine Translation through Phrase Pair Variables
Despite the tremendous success of Neural Machine Translation (NMT), its
performance on low-resource language pairs still remains subpar, partly due to
the limited ability to handle previously unseen inputs, i.e., generalization.
In this paper, we propose a method called Joint Dropout, that addresses the
challenge of low-resource neural machine translation by substituting phrases
with variables, resulting in significant enhancement of compositionality, which
is a key aspect of generalization. We observe a substantial improvement in
translation quality for language pairs with minimal resources, as seen in BLEU
and Direct Assessment scores. Furthermore, we conduct an error analysis, and
find Joint Dropout to also enhance generalizability of low-resource NMT in
terms of robustness and adaptability across different domainsComment: Accepted at MT Summit 202